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SPECIFICATION 

TO WHOM IT MAY CONCERN: 

Be it known that we, with names, residence, and citizenship listed below, have invented 
the inventions described in the following specification entitled: 



INTERNET SERVER SYSTEM TEST AND MEASUREMENT 



Kenneth William Shrum 

Residence: 4255 Fall River Drive, Fort Collins, CO 80526 

Citizenship: United States of America 
Thomas G. Bartz 

Residence: 2014 Agate Court, Loveland, CO 80538 

Citizenship: United States of America 
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INTERNET SERVER SYSTEM TEST AND MEASUREMENT 
Related Applications 

This application is a non-provisional application based on a provisional 
application, Ser. No. 60/200,295, filed 4/28/00. 



Background of the Invention 



[0001] Internet server systems are now a critical component to many successful 

businesses. Many Internet server systems are configured to function as e-commerce 
web sites where computer users can purchase goods and services. The efficient and 
reliable operation of the e-commerce web site is vital to many businesses. 
[0002] In response to the need for efficient and reliable e-commerce web sites, 
test systems have been developed to ensure that the web site is operating within 
tolerable thresholds. These test systems perform automated tests using transactions 
that were previously recorded. To the web site, the test transaction appears like 
another customer. These test user transactions are able to determine how long a 
typical transaction takes and whether or not the e-commerce web site is responding at 
all. 

[0003] Unfortunately, current test systems treat the Internet server system that 
provides the e-commerce web site as a black box - meaning that the test system sends 
in stimulus and measures response. Test systems do not effectively correlate user test 
results with internal performance measurements from the Internet server system. If 
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there is a problem, the test system does not effectively isolate the responsible 
component within the Internet server system. 

[0004] Current test systems also fail to correlate system testing and performance 
data with business performance data. Business performance data may only be 
produced in weekly or monthly reports. If the web site operator receives an alarm from 
a test system, another system must be used to assess the financial damage due to the 
system error. The use of multiple systems is complex and time consuming. 



Summary 

[0007] The invention addresses the above problems with a software solution that 
delivers key business and financial measures in near real time, providing information on 
current ("live") business operations. 

[0008] An e-commerce installation is typically constructed from several tiers. 
These tiers may or may not be physically realized as separate systems. An Internet 
server system would typically comprises a web servers tier, a session servers tier, a 
transaction servers tier and a database servers tier. Each of these tiers potentially 
generates information detailing operations as they are performed. 
[0009] m£if>u§rrton monitors in near real time the business and financial 

performance of the e-commerc^feu^iness. The invention reads the log files from each 
of the tiers, makes queries to databases, arftNnJeracts with the e-commerce 
infrastructure. This information is presented as measurements varying overtime, where 
the measurements are comprised of key business and financial m^asjjres such as the 
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number of new o7<ters]5enT^ of completed orders per minute, the 

number of abandoned "shopping carts" per minute, lost safeTthieJQ^system errors, etc. 
[0010] The invention assesses and reports key business and financial 1 
measurements. The system of the invention delivers these measurements in near-real 
time allowing e-commerce businesses to quickly determine current financial and 
business performance. This in turn allows quick detection and response to variances. 



Description of the Drawings 

FIG. 1 is a diagram illustrating a four tier construction of an Internet server 
system. 



Detailed Description of the Invention 

[001 1] An e-commerce installation is typically constructed from several tiers. 
These tiers may or may not be physically realized as separate systems. A preferred 
embodiment of the invention is shown in FIG. 1. FIG. 1 shows a four tier construction of 
an Internet server system. This construction typically comprises a web servers tier 101 , 
a session servers tier 102, a transaction servers tier 103 and a database servers tier 
104. The web servers 101 are responsible for handling direct customer interaction with 
the e-commerce web pages and delivering content back to the customer. A session 
server 102 keeps track of individual customers and what actions those customers are 
performing. The e-commerce transactions are supported by the transaction servers 103 
and the database servers 104 manage the e-commerce database. Each of these tiers 
potentially generates information detailing operations as they are performed. Also 
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shown in FIG. 1 is a plurality of agents 105. The agents 105 are shown connected to 
each of the elements in the four tiers. For purposes of this example, an agent 105 is 
shown connected to each server in each tier, but it is understood that agents 105 are 
connected to those servers in the tiers 101,102,103,104 as desired by a user. The 
agents 105 are software programs that run on each desired server. Agent programs 
105 are termed "tests". 

[0012] The invention monitors in near real time the business and financial 
performance of the e-commerce business. Applications and computer systems or 
subsystems generate log files in their normal course of operation. For example, a web 
server in the web server tier 101 generates a log file containing hit rates, client/server 
errors, error rates, etc. which can be used to indicate health and performance of the 
server. Generally, log files generated by any subsystem in the tiers 101,102,103,104 
track what occurs in that portion of the system. Accordingly, log files will contain data 
regarding events, state, transactions and anything of interest. It is understood that a log 
file may contain any type or class of data dependent upon programming. 
[0013] Agents 105 associated with the web servers tier 101 are web server log 
readers. Examples of web servers 106 are Netscape, Apache, MicroSoft/IIS, America 
On-Line, etc., and include generic web servers. Events in a web server 106 log file 
include but are not limited to hit rate, hosts served, data volume, error rates, log file size 
and system load. System load includes but is not limited to CPU utilization memory 
usage and I/O transfers. 

[0014] Agents 105 associated with the session servers tier 102 are session 
server application tests and log readers. Examples of sessions servers 107 are session 
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management process tests. Events in a session server 107 log file include but are not 
limited to users served, interaction manager (IM) restarts, IM status, IM load database 
connectivity, error rates and system load. 

[0015] Agents 105 associated with the transaction servers tier 103 are 
transaction server application tests and log readers. Transaction servers 108 process 
information on a variety of levels, including system level performance to transaction 
process count and availability, sales and orders, shopping statistics, payment service 
and order fulfillment (shipping). Events in a transaction server 108 log file may include 
but are not limited to inventory status, system load, open and discarded cart rates, 
shoppers in store, authorized, cancelled and declined data and payment service errors 
Transaction servers 108 also compile and maintain databases. The agents 105 
associated with transaction servers 108 query the transaction server databases using 
standard SQL database queries. It is understood that any type of database query 
capable of obtaining data from a database can be used. Examples of data in a 
transaction server 108 data base may include but are not limited to sales per hour, 
items sold per hour, new order rates, processed order rates, fulfilled order rates orders 
awaiting authorization and orders awaiting shipping. 

[0016] Agents 105 associated with the database servers tier 104 are database 
server tests. Database servers 109 process information and create and maintain 
databases. The agents 105 associated with database servers 109 query the database 
server databases using standard SQL database queries. It is understood that any type 
of database query capable of obtaining data from a database can be used. While any 
data contained in the database can be retrieved, for the examples used in accordance 
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with the invention, the pertinent data to query is health information such as number of 
the transactions that have occurred, the number of users, sales volume and other 
application or business health indicators of the e-commerce website which the database 
may contain. 

[0017] In the preferred embodiment of the invention shown in FIG. 1 , each agent 

105 is shown as being associated with a particular server. The agents are software 

programs, or tests, that are resident on the servers due to the need for direct access to 

the relevant log files or databases. It is understood, that a particular agent 105 will be 

programmed to read the format, or formats, of log files and/or databases resident on a 

particular server. For example, if the agent 105 is interested in events relating to sale 

amount, it knows that column four (4) of the log file contains sale amount data. The 

* 

agent 105 retrieves the appropriate data from the log files or databases and sends that 
information to a user site. The user site is an Internet service management solution 
such as the Firehunter® products offered by Agilent Technologies that enables a user 
to measure and monitor an e-commerce solution's performance. It is understood that 
the invention is not limited to use with Firehunter® and can be used in any manner 
capable of displaying data. 

[0018] The data from the agents 105 is sent to a user site via means capable of 
transferring data. Preferably, the data is transmitted via the Internet or an intranet. It is 
understood that any network means can be used to transmit data from the agents 105 
to a user site. It is also considered that a direct connection be used to transmit data 
from the agents to the user site. 
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[0019] \jhe invention reads the log files from each of the tiers 101 ,102,103,104 in 
which there is arKagent, makes queries to the database, and interacts with the e- 
commerce infrastructure. This information is presented as measurements varying 
overtime, where the measurements are comprised of system infrastructure (such as 
load or error information) and Kfey business and financial measures such as the number 
of new orders per minute, the dollarVqlume of completed orders per minute, the number 
of abandoned "shopping carts" per minute>etc. Gathering these infrastructural and 
business measurements together into a systenvK^e Firehunter® then enables 
correlated results, such as lost sales due to system errqrs and the impact of system 
load on the rate and number of business transactions. 
[0020] The invention assesses and reports key business and financial 
measurements. The system of the invention delivers these measurements in real time 
allowing e-commerce businesses to quickly determine current financial and business 
performance. This in turn allows quick detection and response to variances. 
[0021] It is understood that the invention is not limited to the specific examples 
and illustrations discussed above, but only by the following claims and their equivalents. 
Accordingly, the invention is applicable to a broad variety of log files and databases that 
contain information that has been logged by an application. 
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